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(57) A vehicle environment monitoring system for an 
automotive vehicle, is capable of accurately detecting 
the movement of an object existing in an external envi- 
ronment of the vehicle, and determining the probability 
of collision between the object and the vehicle, thereby 
appropriately warning the driver A running speed of the 
vehicle is detected, and a distance between the object 
and the vehicle Is calculated, based on the image ob- 
tained by the Imaging device. A relative speed between 
the object and the vehicle Is calculated according to the 
calculated distance. It is detennined whether or not 
there is a significant probability that the vehicle may col- 
lide with the object, based on the running speed of the 
vehicle and the distance therebetween on condition that 
the relative speed is in the vicinity of the running speed 
of the vehicle. 
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Description 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

[0001] The present invention relates to a vehicle environment monitoring system that monitors an external environ- 
ment of a vehicle on which the system is installed. The system detects an external object, which may collide with the 
vehicle from an image obtained by an imaging device mounted on the vehicle. The system hereof assists a user In 
10 avoiding a potential collision against a large animal, such as a deer, a bear or the like, since such collision has an 
adverse influence on the vehicle. 

Description of the Prior Art 

75 [0002] Conventionally, a warning device for giving a warning when the distance between the vehicle and an object 
approaching the vehicle becomes short was proposed by Japanese Laid-Open Patent Publication No. 2001 -6096. The 
device detects a distance between the vehicle and the object based on the displacement between images of the object, 
that is, the parallax of images obtained by two cameras mounted on the vehicle. According to this device, the change 
rale of the detected distance is calculated as a relative speed, and a time period TY until the estimated time the vehicle 

20 may collide with the object (an estimated allowance time period TY) Is calculated based on the relative speed and the 
distance. Further, a warning is given when the estimated allowance time period TY is shorter than a predetermined 
allowance time period T. 

[0003] FIG. 22 is a diagram showing actually detected data of an error of the relative speed detected by the method 
shown in the above publication. As is clear from FIG. 22, the error of the detected relative speed increases as the 
25 distance from the object increases. Therefore, calculating the estimated allowance time period TY based on the relative 
speed and the distance raises a problem that the warning may be given too eariy or too late. 

[0004] For example, if the detected relative speed is 60 km/h at a distance of 60 meters, the estimated allowance 
time period TY is 3.6 seconds (=(60/60000) x 3600). Accordingly, if the predetermined time period T is set to 4 seconds, 
a warning is given. However, if the relative speed is erroneously detected as 40 km/h at the distance of 60 meters, the 

30 estimated allowance time period TY becomes 5.4 seconds (=(60/40000) x 3600). Then a warning Is not given although 
the probability that the collision may occur after 3.6 seconds is high. Even in this case, a warning is given after a little 
while, since the accuracy of detecting the distance becomes higher as the distance decreases. That is, the timing of 
giving a warning is delayed. In contrast, If the relative speed is erroneously detected as higher than the actual speed, 
the timing of giving a warning becomes too eariy. 

35 [0005] Further, the above-described conventional method causes another problem that the vehicle running in the 
opposite lane and approaching at a comparatively high speed, or the vehicle running ahead In the same lane may be 
detected as an object with high possibility of collision, which often causes unnecessary wamings. 

SUMMARY OF THE INVENTION 

40 

[0006] It is an object of the invention to provide a vehicle environment monitoring system, which is capable of more 
accurately determining the probability of collision between the object and the vehicle, thereby giving a waning to the 
driver at a desired timing, and preventing unnecessary warnings. 

[0007] To attain the above object, the present invention provides a vehicle environment monitoring system, which 
45 detects an object existing In an external environment of the vehicle from an image, obtained by imaging means mounted 
on the automotive vehicle. The vehicle environment monitoring system includes vehicle speed detecting means, dis- 
tance calculating means, relative speed calculating means, and probability determining means. The vehicle speed 
detecting means detects a running speed of the vehicle. The distance calculating means calculates a distance between 
the object and the vehicle, based on the image obtained by the imaging means. The relative speed calculating means 
so calculates a relative speed between the object and the vehicle, according to the distance calculated by the distance 
calculating means. The probability detemnining means determines whether or not there is a substantial probability that 
the vehicle may collide with the object, based on the running speed of the vehicle and the distance calculated by the 
distance calculating means. A warning alarm is generated, provided on condition that the relative speed between the 
object and the vehicle Is in the vicinity of the running speed of the vehicle. 
55 [0008] According to this vehicle environment monitoring system, it is detemnined whether or not there is a significant 
probability that the vehicle may collide with the object, based on the running speed of the vehicle instead of the relative 
speed, and the distance between the vehicle and the object. This probability is determined on condition that the relative 
speed is in the vicinity of the running speed of the vehicle, In other word, the sensed object is moving at a relatively 
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low speed, or standing still. Therefore, It is possible to issue a warning at a desired tinning, even if the detection error 
of the relative speed is relatively large. Further since the detennnination is carried out on condition that the relative 
speed is in the vicinity of the vehicle speed, it is possible to substantially prevent unnecessary warnings when the 
vehicle is approaching another vehicle that is running in the opposite lane, or running ahead in the same lane. 
5 [0009] Preferably, the detennining means carries out the detemnination, when an absolute value of a difference 
between the relative speed and the running speed of the vehicle is less than or equal to half of the running speed of 
the vehicle. 

[0010] Preferably, the vehicle environment monitoring system further includes relative position-detecting means and 
movement vector-calculating means. The relative position-detecting means detects a relative position of the object to 

10 the vehicle, based on the image obtained by the imaging means and the distance calculated by the distance calculating 
means, to thereby obtain position data. The movement vector-calculating means calculates positions of the object in 
a real space, based on a plurality of time series items of the position data detected on the object by the relative position- 
detecting means, and calculates a movement vector of the object based on the positions in the real space. The prob- 
ability determining means determines whether or not the probability of collision is high based on the movement vector, 

15 when it is determined based on the mnning speed of the vehicle and the distance that there is a possibility of collision. 
[0011] According to this configuration, an accuracy of the detemiination is improved by using the movement vector. 
[0012] Preferably, the movement vector-calculating means includes approximate straight line-calculating means for 
calculating an approximate straight line approximating a locus of relative movement of the object, and position data- 
correcting means for correcting the time series items of the position data by using the approximate straight line. The 

20 movement vector-calculating means calculates the movement vector based on the corrected time series items of the 
position data. 

[0013] According to this configuration, a detection error relating to position data is reduced, to make it possible to 
more accurately determine the probability of collision. 

[001 4] Preferably, the probability detennining means carries out the detenmination by applying collision detemni nation 
25 conditions that are dependent on a width of the vehicle. 

[0015] According to this configuration, the probability of collision is more accurately determined and unnecessarily 
alamis are prevented. 

[0016] More preferably, the approximate straight line-calculating means calculates the approximate straight line as 
a three-dimensional straight line including data of a height of the object. 
30 [0017] According to this configuration, it is possible to accurately detemiine the probability of collision, even when 
the vehicle is running on a hilly road. 

[0018] Preferably, the imaging means comprises two infrared cameras capable of detecting infrared rays. 

[0019] According to this configuration, an animal, a running automotive vehicle, or the like can easily be detected, 

even when the driver is driving at night and it is difficult to recognize them. 

35 [0020] Preferably, the imaging means comprises two TV cameras for detecting infrared rays or visible rays, and the 
relative position-detecting means includes search area-setting means and con-esponding object image-identifying 
means. The search area-setting means sets, based on a position of an object image contained in an image output 
from one of the two TV cameras, a search area within an Image output from the other of the two TV cameras, for 
searching for a corresponding object image contained in the image output from the other TV camera. The corresponding 

40 object image-identifying means identifies the corresponding object image by carrying out a correlation operation on 
data within the search area. The distance-calculating means calculates the distance between the vehicle and the object, 
based on a parallax between the first object Image and the corresponding object image. 

[0021] Preferably, the vehicle environment monitoring system includes warning means for warning a driver, when it 

is determined by the detennining means that there is a high probability of collision against the object. The warning 
45 means is preferably Inhibited from warning the driver when the driver Is carrying out a braking operation, and at the 

same time, deceleration caused by the braking operation Is larger than a predetennined threshold. 

[0022] According to this configuration, the driver is not warned when he has already recognized the object and is 

carrying out an appropriate braking operation. This makes it possible to substantially prevent the driver from being 

annoyed by an unnecessary waming. 
50 [0023] The above and other objects, features, and advantages of the invention will become more apparent from the 

following detailed description taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

55 [0024] FIG. 1 is a block diagram showing the arrangement of a vehicle external environment monitoring system 

according to an illustrative embodiment of the invention; 

[0025] FIG. 2 is a diagram illustrating portions of a vehicle at which mounting cameras appearing in FIG. 1 are 
mounted; 
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[00261 FIG. 3 is a flowchart showing a procedure of processing steps executed by an image-processing unit appear- 
ing in FIG. 1; . 
[0027] FIG 4 is a flowchart showing details of a warning determination process in FIG. 3; 

[0028] FIGS. 5A and 5B are diagrams each showing halftone portions as hatched areas, which are useful in explaining 
5 gray scale images obtained by infrared cameras, In which: 

[0029] FIG. 5A shows a right image obtained by a right one of the infrared cameras; 
r00301 FIG 5B shows a left image obtained by a left one of the same; 

[0031] FIG. 6 is a diagram showing a black area as a hatched one. which is useful in explaining an image formed 

by binarlzatlon of a gray scale image; , ^- u- H^ta tn 

10 [0032] FIGS. 7A to 7C are diagrams which are useful in explaining a process for converting binanzed image data to 
run length data and labeling of objects for identification thereof; 

[0033] FIGS. 8A and 8B are diagrams useful in explaining how objects are tracked at time intervals; 
[0034] FIG. 9A is a diagram useful In explaining a target Image In the right Image; 
r00351 FIG 9B is a diagram useful in explaining a search area set In the left image; 
15 [0036] FIG. 1 0A and 1 08 are diagrams useful in explaining a correlation operation carried out on data within the 

search area; . 

r00371 FIGS 1 1 A and 1 1 B are diagrams useful In explaining a method of calculating a parallax; 

[0038] FIG. 12 is a diagram useful in explaining a method of calculating a distance between the vehicle and the 

object based on the parallax; _,. ^. , ^, 

20 [0039] FIGS 1 3A and 1 3B are diagrams useful for explaining a coordinate system used in the present embodiment. 

[00401 FIG 1 4 is a diagram useful In explaining a turn angle-dependent correction of position data of an object; 

[0041] FIG. 15 is a diagram showing the displacement of the object in the images due to tuming of the vehicle; 

[0042] FIG 1 6 is a diagram useful in explaining a method of calculating a relative movement vector; 

[0043] FIG. 17 is a diagram useful in explaining conditions for determining whether or not a warning should be issued; 

25 [0044] FIG. 1 8 is a diagram useful in explaining divisional areas In front of the vehicle; 

[0045] FIG 1 9 is a diagram useful in explaining a case in which a collision is liable to occur; 

[0046] FIG. 20 is a diagram useful in explaining a method of incoming object collision detennination dependent on 

a width of the vehicle; 

[0047] FIG. 21 A shows a state where a screen of a head up display Is not displayed; 

30 [0048] FIG. 21 B shows a state where a screen of the head up display is displayed; and 

[0049] FIG. 22 is a diagram for explaining a problem associated with the conventional technique. 

DETAILED DESCRIPTION 

35 [0050] The invention will now be described in detail with reference to drawings showing an embodiment thereof. 
[00511 Referring first to FIG 1 , there is shown the arrangement of a vehicle environment monitoring system, accord- 
ing to an illustrative embodiment of the Invention. The system depicted in FIG. 1 includes right and left infrared cameras 
1 R 1 L capable of detecting far-infrared rays, a yaw rate sensor 5 for detecting yaw rate of the vehicle, a vehicle speed 
ser^sor 6 for detecting traveling speed (vehicle speed) VCAR of the vehicle, a brake sensor 7 for detecting an operation 
amount of a brake (not shown), and an image-processing unit 2 for detecting an object, such as an anirrial or the like, 
ahead of the vehicle. Objects are detected based on image data obtained by the above cameras 1 R, 1 L. The system 
also includes a speaker 3 for generating a voice alarm for warning the driver, and a head up display (hereinafter refen-ed 
to as the "HUD") 4 for displaying an image obtained by the camera 1 R or 1 L. The HUD 4 pemiits the driver to recognize 
the object having the high probability of collision against the vehicle. ^. . 

[00521 As shown In FIG. 2, the cameras 1 R, 1 L are an^nged In afront portion of the automotive vehicle 1 0 at locations 
symmetric with respect to the longitudinal central axis of the vehicle 10. The cameras are rigidly fixed to the vehicle 
such that the two cameras 1 R, 1L have optical axes In parallel with each other and disposed at equal heights frorn a 
road surface. The infrared cameras 1 R, 1L have a characteristic that the output signal level thereof becomes higher 
(the luminance of an image of an object increases) as the temperature of the object becomes higher. 
so 100531 The image-processing unit 2 includes an A/D converter circuit for converting input analog signals to digital 
signals, an lmagememoryforstorlngdlgitizedimagesignals.aCPU(Central Processing Unit)forcarryingoutanthmet.c 

operations, a RAM (Random Access Memory) used by the CPU for storing data being processed in the anthmetio 
operations, a ROM (Read Only Memory) storing programs executed by the CPU, tables, and maps, and an output 
circuit for outputting driving signals to the speaker 3. display signals to the HUD 4, and the like. Output signals from 
55 the cameras 1R,1 Land the sensors 5 to 7 are converted to digital signals and input to the CPU. 

[0054] As shown in FIG. 2, the HUD 4 is anranged such that a screen 4a thereof is displayed in a front window at a 
location ahead of the driver. . 
[0055] FIG. 3 is a flowchart showing a procedure of processing steps executed by the image-processing unit 2. First. 
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output signals from the cameras 1 R, 1L are subjected to A/D conversion by the A/D converter circuit and the resulting 
digital data are stored in the image memory (steps S11 , SI 2, SI 3). Data of images stored in the Image memory is data 
of gray scale images including luminance infonmation. FIGS. 5A and 5B are diagrams for explaining gray scale images 
obtained by the respective cameras 1 R. 1 L (a right image by the camera 1 R. and a left image by the camera 1L). 

5 Hatched areas in the right and left images are halftone (gray) areas, while areas surrounded by thick solid lines are 
areas at a high luminance level (at a high temperature). The areas surrounded by thick, solid lines are areas (hereinafter 
referred to as "high luminance areas") of detected objects, displayed in white on the screen. In the right image and the 
left image, an identical object is displayed as dual, respective images at respective locations horizontally displaced 
from each other, so that it is possible to calculate a distance from the vehfele 10 to the object, based on the image 

10 displacement (parallax). 

[0056] At step S14 in FIG. 3, the right image is set to a reference image, and the digital image data representative 
of the reference image is binarized (converted to 1 -bit data) such that an area at a level of luminance equal to or higher 
than a luminance threshold ITH, experimentally detennined, is set to "1 " (white) and an area at a lower level of luminance 
than the threshold ITH is set to "0" (black). FIG. 6 shows an image obtained by binarization of the FIG. 5A image. In 

15 the figure, a hatched area represents a black area, while areas sun^ounded by thick solid lines (high luminance areas) 
represent respective white areas. 

[0057] At the following step S15, the binarized image data is encoded Into run length data. FIG. 7A Is a diagram for 
explaining the encoding process. In the figure, areas set to white by the above binarization are represented by lines 
LI to L8 indicative of respective lines of pixels. Although the lines LI to L8 each have a width of one pixel in the y 

20 direction, and are actually arranged side by side without any gaps in the y direction, they are shown as separate lines 
spaced from each other for clarity of description. Further, the lines L1 to L8 have respective lengths of two pixels, two 
pixels, three pixels, eight pixels, seven pixels, eight pixels, eight pixels, and eight pixels in the x direction. The run 
length data represents each of the lines L1 to LB by the coordinates of the starting point (point of the left end) of the 
line and the length (the number of pixels) of the line from the starting point to the ending point (point of the right end) 

25 of the same. For instance, the line L3 is formed of three pixels (x3, y5), (x4, y5) and (x5, y5), and represented by the 
run length data (x3, y5, 3). 

[0058] At steps SI 6 and S17, an object is extracted by labeling the same, as shown in FIG. 7B. That is, out of the 
lines L1 to L8 encoded into the run length data, the lines Li to L3 overlapping in the y direction are regarded as one 
object 1 . and the lines L4 to L8 overlapping in the y direction are regarded as one object 2, whereby object labels 1 
30 and 2 are added to the run length data. This processing enables, e.g. the high luminance areas appearing In FIG. 6, 
to be grasped as objects 1 to 4, respectively. 

[0059] At step 81 8, as shown in FIG. 7C, the centroid G of an extracted object (image of an object), the area S of 
the extracted object, and the aspect ratio ASPECT of a rectangle circumscribing the extracted object (indicated in FIG. 
7C by broken lines) are calculated. The area S is calculated by integrating the lengths of the run length data of an 
35 identical object. The coordinates of the centroid G is calculated as the x coordinate of a line equally dividing the area 
S along the y direction into halves, and the y coordinate of a line equally dividing the area S along the x direction into 
halves. The aspect ratio ASPECT is calculated as Dy/Dx which is a ratio of Dy to Dx appearing in FIG. 7C. It should 
be noted that the position of the centroid of the circumscribing rectangle nnay be used in place of that of the centroid 
G of the extracted object. 

40 [0060] At step SI 9, objects are tracked at time intervals, that is, identification or recognition of identical objects is 
carried out whenever each sampling repetition period elapses. Assuming that a time obtained by discretizing time t as 
an analog amount by a sampling repetition period is represented by k, objects 1 and 2 extracted at time k, as shown 
in FIG. BA, and objects 3 and 4 extracted at time (k + 1) as shown in FIG. 8A are checked as to their identity. More 
specifically, it is detemnined that the objects 3 and 4 are identical with the respective objects 1 and 2 when the following 

45 identification conditions (1 ) to (3) are satisfied, and the objects 3 and 4 are labeled as objects 1 and 2 to thereby track 
the objects at time intervals: 

[0061] (1) Assuming that the position coordinates of the centroid of objects i (= 1, 2) in an image at time k are 
represented by (xi(k), yi(k)), and the position coordinates of the centroid of objects j (= 3, 4) in an image at time (k + 
1) are represented by (xj(k + 1), yj(k + 1)), the following conditions are satisfied: 



so 



55 



|xj(k+1)-xl(k)|<Ax, 

and 

|yj(k+1)-yi(k)|<Ay 
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15 



Wherein Ax and Ay represent maximum altowable values for respective distances covered by the object moving in the 
image in the X direction and the y direction. ^L.,ei/i.\ <^^Mtha 

[0062] (2) Assuming that the areas of the objects i (= 1 , 2) in the image at time l< are represented by SKk), and the 
areas of the objects j (= 3 , 4) in the image at time (k + 1 ) are represented by Sj(k + 1 ), the following condition is satisfied: 

Sj(k + 1)/Si(k)<1 ± AS 

wherein AS represents a maximum allowable value for a change In the area. 

[00631 (3) Assuming thatthe aspect ratios of rectangles circumscribing the objects i (= 1 , 2) at time k are represented 
by ASPECT i(k), and the aspect ratios of rectangles circumscribing the objects j (= 3. 4) at time (k + 1 ) are represented 
by ASPECT j(k + 1), the following condition is satisfied: 

ASPECT j(k + 1)/ ASPECT i(k) < 1 ± AASPECT 



20 



25 



Wherein AASPECT represents a maximum allowable value for a change in the aspect ratio. 

[00641 When comparison is made between FIG. 8A and FIG. 8B, the size of each object is increased in FIG. 8B, but 
the objects 1 and 3 satisfy the above identification conditions, and the objects 2 and 4 satisfy the above identification 
conditions Hence, the objects 3 and 4 can be identified with the respective objects 1 and 2. The position coordinates 
(of the centroid) of each object thus identified are stored in the memory as time series items of position data, and used 
for carrying out subsequent arithmetic operations. , 
[0065] It should be noted that the above-mentioned processes at steps S1 4 to S1 9 are executed on the binarized 
reference image (the right image in the present embodiment). 

[0066] At step S20 in FIG. 3, the vehicle speed VCAR detected by the vehicle speed sensor 6 and the yaw rate YR 
detected by the yaw rate sensor 5 are read in for integration of the yaw rate YR over time, whereby the angle er of turn 
of the automotive vehicle 1 0 (see FIG. 1 4) Is calculated. 
[0067] On the other hand, at steps S31 to S33, an operation for calculating a distance z between the object and the 
automotive vehicle 1 0 is carried out in parallel with the processes at steps S1 9 and 20. This arithmetic operation takes 
30 a longer time period than the processes at steps 81 9 and 20, and hence it is executed at a longer repetition period 
than that of the processes at steps S 1 9 and 20 (approximately three times as long as a repetition period of execution 
of the processes from step S11 to step S20). , . . » . x • 

[0068] At step S31 , one of objects tracked by using the binarized image of the reference image (nght image) is 
selected whereby as shown in FIG. 9A, a target image R1 (whole area surrounded by a circumscnbmg rectangle is 
set to a target image in this example) is extracted from the right Image. At the following step S32. a search area for 
searching an Image (hereinafter referred to as the "corresponding Image") corresponding to the target image is set 
within the left image, and the corresponding image is extracted by performing a correlation operation. More specifically, 
as shown in FIG 9B, a search area R2 is set in the left image based on the coordinates of each vertex of the target 
Image R1 , and the total sum value C (a. b) of luminance differences indicative of a degree of correlation with the target 
Image R1 is calculated as to data within the search area R2 by using the following equation (1 ). whereby an area which 
provides the minimum value of the total sum value C (a. b) is extracted as the corresponding image. 
[0069] It should be noted that the correlation operation is carried out not by using data of binarized images (binary 
data) but by using data of gray scale images (gray scale data). Further, when position data of an identical object 
obtained in the past is available, an area R2a (shown by broken lines in FIG. 9B) narrower than the search area R2 is 
45 set to the search area based on the position data. 

C(a, b)= 's^'s^ I IL(a+m-M. b+n-N)-IR(m, n) I . (D 
n=0 m=0 

wherein IR (m n) represents a luminance value of a position shown by coordinates (m. n) in the target image R1 
appearing In FIG 10A. and IL(a ^- m - M, b -.- n - N) represents a luminance value of a position shown by coordinates 
(m n) in a local area R3 having the same shape as that of the target image R1 with a reference point (origin of its own 
coordinate system) set to coordinate (a, b) in the search area R2 as indteated in FIG. 10B. A position minimizing the 
total sum value C (a. b) of the luminance differences is obtained by changing the coordinates (a, b) of the reference 
point whereby the position of the corresponding image can be determined. 

[0070] After the target image R1 and the corresponding image R4 conresponding to this target image R1 as shown 
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in FIG. 11A and 11B are extracted at step S32. then at step S33, a distance dR (the number of pixels) between the 
centroid of the target image R1 and a center line LCTR of the right image, and a distance dL (the number of pixels) 
between the centroid of the corresponding image R4 and a center line LCTR of the left image are determined, and 
applied to the following equation (2) to calculate a distance z between the automotive vehicle 10 and the object. 

5 

- B X F _ B X F /2) 
^ " (dL + dR) X p " Ad X p 

wherein B designates the length of a baseline, that is, a horizontal distance (in the x direction) between the center 
10 position of an imaging element 11 R of the camera 1 R and the center position of an Imaging element 1 1 L of the camera 
1L (i.e. distance between the optical axes of the respective cameras), as shown in FIG. 12; F designates the focal 
distance of lenses 12R, 12L; p designates a space interval between pixels in the imaging elements 11 R, 11 L; and Ad 

(= dR + dL) represents the amount of parallax. 

[0071] At step S21 , coordinates (x, y) in the image and the distance 2 calculated by using the equation (2) are applied 
15 to the following equations (3) for conversion to real space coordinates (X, Y, Z). Here, the coordinate system of the 
real space coordinates (X, Y, Z) (real space coordinate system) is defined as shown in FIG. 13A with the position of a 
mid point (position fixed on the automotive vehicle 1 0) of mounting positions at which the cameras 1 R, 1 L are mounted 
being set to an origin O of the real space coordinate system, and coordinates In a coordinate system of an image 
(imaginary image, refen^ed to hereinafter) corresponding to the real coordinate system is defined as shown in FIG. 
20 13B, with the center of the Image being set to the origin of the coordinate system corresponding to the real space 
coordinate system, the horizontal direction being defined as the x direction, and the vertical direction being defined as 
the y direction. 



X 




xc X z/i 




Y 




yc X z/f 


(3) 


Z 




z 





30 

f=Fyp 

[0072] In these equations, (xc, yc) are obtained by converting the coordinates (x, y) in the coordinate system of the 
35 right image to coordinates in the coordinate system of the imaginary image, the center of which is caused to coincide 
with the origin O of the real space coordinate system, based on the relative relationship of the mounting position of the 
camera 1 R and the origin O of the real space coordinate system. Further, f represents a ratio between the focal distance 
F and the pixel-to-pixel space interval p. 

[0073] At step S22, a turn angle-dependent correction of position data of each object is carried out by compensating 
40 for positional displacement of the object in the image due to tuming of the vehicle 1 0, according to the angle of turn. 
Assuming that the vehicle 10 makes a turn, as shown in FIG. 14, e.g. in a leftward direction through an angle 6 r of 
turn during a time period from a time k to a time (k + 1), an image obtained by the camera is displaced by Ax in the x 
direction, as shown in FIG. 1 5. The turn angle-dependent correction at step S22 is carried out to compensate for the 
displacement of the image. More specifically, the real space coordinates (X, Y, Z) are applied to the following equation 
45 (4) to calculate corrected coordinates (Xr, Yr, Zr). Real space position data (Xr, Yr, Zr) thus calculated is stored in the 
memory in a manner correlated to each corresponding object. Further, In the following description, the corrected co- 
ordinates are denoted as coordinates (X, Y, Z). 



Xr 
Yr 
Zr 

55 

[0074] At step S23. as shown in FIG. 16, an approximate straight line LMV con-esponding to a relative movement 
vector between an identical object and the automotive vehicle 10 is obtained based on N real space position data 



cos 0 r 0 — sin0r X 
0 10 Y 
sin 6 T 0 cos Or Z 



(4) 
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items, i.e. time series position data (N is a number equal to or close to 10. for instance) after the turn angle-dependent 
correction, which were obtained during a time period AT. More specifically, assuming that a direction vector indicative 
of the direction of the approximate straight line LMV is represented by L = (Ix, ly. Iz) (ILI = 1 ). a straight line represented 
by the following equations (5) is determined: 

X=u-lx+Xav 

Y=u-ly+Yav (5) 
Z=u*lz+Zav 



15 



N-1 

Xav = I X(j)/N 
j=0 



20 



25 



SO 



N-1 

Yav = 2 Y(j)//N 
j=0 



N-1 

Zav = 2 Z(j)/N 



wherein u represents a parameter capable of having an arbitrary value, and Xav, Yav and Zav represent the average 
value of X coordinates, the average value of Y coordinates, and the average value of Z coordinates, respectively, of 
series of real space position data. Further, if the parameter u is eliminated from the equations (5), the following equation 
30 (5a) is obtained: 

(X - Xav)/lx = (Y - Yav)/ly = (Z - Zav)/lz (5a) 

35 [0075] FIG. 16 is a diagram illustrating the approximate straight line LMV. In the figure, P(0), P(1 ), P(2) , P(N - 

2), P(N - 1 ) designate points indicated by respective time series position data items (data points) after the turn angle- 
dependent correction. The approximate straight line LMV is detemriined as a straight line which extends through the 
average position coordinates Pav (= Xav, Yav, Zav) of the time series position data item, and minimizes an average 
value of the squares of respective distances between the line and the data points. Here, a numeric value in parentheses 

40 added to each P indicating the coordinates of each data point (position coordinates) shows that the larger the numeric 
value is, the older the data item having the numeric value Is. For instance, P(0) indicates the latest position coordinates, 
P(1 ) indicates position coordinates obtained one sampling repetition period earlier, and P(2) indicates position coordi- 
nates obtained two sampling repetition periods earlier. The same applies to DO), XQ), YQ), ZQ) and the like referred to 
hereinafter 

45 [0076] More specifically, inner products s of vector DQ) = (DX(J). DY(i), DZQ)) = (XQ) - Xav, YQ) - Yav, Z(i) - Zav) which 
extend from the average position coordinates Pav to the coordinates P(0) to P(N - 1 ) of the respective data points and 
a direction vector L are calculated by the following equation (6), and the direction vector L = (Ix, ly, Iz) maximizing the 
variance of the inner products s is detemnined. 



s = 1x • DXO) + 1y • DYQ) + 1z • DZQ) (6) 



[0077] A variance-covariance matrix V of the coordinates at the respective data points is represented by the following 
equation (7). Since a characteristic value a for this variance-covariance matrix V corresponds to the variance of the 
55 inner products s, a characteristic vector corresponding to the largest one of three characteristic values calculated from 
the matrix becomes the direction vector L desired to be obtained. It should be noted that in order to calculate charac- 
teristic values and a characteristic vector from the matrix of the equation (7), a method known as Jacobian method 
(described e.g. in "Suuchi-Keisan Handbook (Handbook of Numeral Calculus)" (published by Ohmsha, Ltd, Tokyo, 
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V = 



N-l 



N-l 9 
Z DXO) ^ 

Z DXO)-DYO) 

'2^Dxa)-Dza) 



N-l 

2 DXG)-DYG) 

N-l o 

Z DYG) ^ 

j=o 

^Z DYO)-DZ(j) 



N-l 

Z DXC)-DZO) 
*Z^ DYG)-DZG) 

N-l 9 

Z DZG) 
j=o 



(7) 
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20 



25 



30 



[0078] Next, the latest position coordinates P(0) = (X(0), Y(0), 2(0)), and the position coordinates P(N - 1) = (X(N - 
1). Y{N - 1), Z(N - 1)) obtained (N - 1) sampling repetition periods (time period AT) earlier are corrected such that 
positions thereof are on the approximate straight line LMV. More specifically, by applying the Z coordinates, Z(0), Z(N 
- 1) to the above-mentioned equation (5a), that is, by using the following equations (8), corrected position coordinates 
Pv(0) = (Xv(0), Yv(0), Zv(0)) and corrected position coordinates Pv(N - 1 ) = Xv(N - 1), Yv(N - 1),Zv(N-1) are calculated. 



XvO) = (Z(j)-Zav)X 
Yv(j) = (Z(j)-Zav) X 

Zva)=Z(j) 
j=0, N-l 



Ix 
Iz 

Iz 



-Xav 
-Yav 



(8) 
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[0079] As the vector extending from the position coordinates Pv(N - 1) to the position coordinates Pv(0) calculated 
by the equations (8), the aforementioned relative movement vector can be obtained. As described above, an approx- 
imate straight line approximating the locus of relative movement of an object to the automotive vehicle 1 0 is calculated, 
based on a plurality of (N) data items of position data during a monitoring time period AT, and a relative movement 
vector is determined, based on the approximate straight line. This makes it possible to reduce adverse influence of 
position detection enters, and more accurately estimate the probability of collision against the object in advance. 
[0080] Now, refen-ing again to FIG. 3, at step S24, a warning determination process (FIG. 4) is earned out by deter- 
mining whether or not there is a possibility of collision against the detected object, and issuing a warning if the probability 
of the collision is high. 

[0081] At step S41 in FIG. 4, a relative speed Vs in the Z direction is calculated by the following equation (9). When 
the following expressions (10a), (10b), and (11) are satisfied, It is detennined that there is a significant probability of 
collision, and the program proceeds to step S42. If any one or more of the expressions (10a), (10b), and (11) is/are 
not satisfied, the warning determination process is terminated. 



Vs=(Zv(N-1)-Zv(0))/AT 



(9) 



so 



VCAR/2 ^ Vs ^ VCAR x 3/2 



(10a) 
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(I VCAR - Vs| ^ VCAR/2) 
Zv(0)/VCAR ^ T 



(10b) 
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|Yv(0)| S H O"") 

[0082] In the above expressions. Zv(0) represents the latest detected distance value (although v is added to the 
5 symbol to indicate that the symbol represents a value of a data item corrected by using the approximate straight line 
LMV the Z coordinate itself has the same value as it has before the con-ection). and Zv(N - 1) represents a distance 
value detected a time period AT earlier. Further, T represents a lead time provided so as to enable detemnining the 
possibility of collision a time period T earlier than an estimated collision time. T is set to a time period within a range 
of e g two to five seconds. Further, H represents a predetermined height for defining a range in the Y direction, that 
10 is, in the direction of height, which is set to e.g. a height approximately two times as large as that of the automotive 
vehicle 1 0 

[0083] The expression (1 Da) is satisfied when the relative speed Vs Is In the vicinity of the vehicle speed VCAR, in 
other words when the sensed object is moving at a relatively low speed, or standing still. 

[00841 The relationship implied in the expression (1 Ob) can be shown by FIG. 1 7. When all of the following conditions 

15 are satisfied, determination processes at step 42 et seq. are carried out: 

[0085] 1) the expression (10a) is satisfied, i.e., the relative speed Vs is in the vicinity of the vehicle speed VCAR; 
[0086] 2) the expression (1 Ob) Is satisfied, i .e, , coordinates corresponding to the relative speed Vs and the distance 
Zv(0) both detected or calculated are within an hatched area in FIG. 1 7; and 
[0087] 3) the expression (11) is satisfied, i.e., lYv(O) I is less than or equal to H. 

20 [0088] A significant probability of collision is determined on condition that the expressions (1 Oa) and (1 Ob) are sat- 
isfied and the warning is issued according to the position and the relative movement vector of the object. Therefore, 
even if a detection error of the relative speed is comparatively large, the probability of collision is accurately determined, 
and the warning can be issued at a desired timing. Further, other vehicles that are running in the opposite lane, or 
running ahead in the same lane, are excluded from the objects of warning, by the condition of the expression {10a). 

25 Accordingiy, frequent unnecessary warnings can be substantially prevented. 

[0089] In FIG 1 8, an area which can be monitored by the cameras 1 R, 1 L is shown by an area ARO defined by an 
outertriangle indicated by thick solid li nes. In the area ARO, areas AR1 , AR2. and AR3 which are closer to the automotive 
vehicle 10 than Z1 = Vs x T (vehicle speed x lead time) are set to warning detennination areas. Here, the area AR1 is 
an area corresponding to a range including the width a of the automotive vehicle 1 0 and allowances p, p (each having 

30 a value of e.g. 50 to M 00 cm) provided on opposite sides of the vehicle 1 0, in other words, an area having a width of 
(a /2 + p) on each side of the central axis in the longitudinal direction of the vehicle 1 0. If the object continues to exist 
in this area there is a very high probability of collision against the vehicle 1 0. Hence, hereinafter, this area is refenred 
to as the "closing object detennination area". The areas AR2, AR3 are areas having X coordinates with larger absolute 
values than those of X coordinates in the closing object determination area (areas laterally outward of the closing object 

35 determination area). An object in one of these areas AR2 and AR3 is subjected to an incoming object collision deter- 
mination process described hereinafter, for determining the possibility of potential collision of an incoming object. These 
areas AR2 and AR3 are referred to as the "incoming object determination areas". Further, the above areas have the 
predetermined height H in the Y direction, as shown in the expression (11). 

[0090] The answer to the question of step S41 becomes affirmative (Yes) when the object exists in the closing object 
40 determination area AR1 or in one of the incoming object detemiination areas AR2, AR3. 

[0091] At the following step S42, it is determined whether or not the object exists in the closing object determination 
area AR1 . If the answer to the question of step S42 Is affimnative (Yes), the program immediately proceeds to step 
S44 whereas if the answer is negative (No), the incoming object collision determination process for determining the 
possibility of potential collision of the incoming object is carried out at step S43. More specifically, it is determined 
43 whether or not the following expression (12) is satisfied by the difference between the latest x coordinate xc(0) (as 
described above, c is added to the symbol to indicate that the symbol represents the x coordinate corrected by causing 
the center position of the image to coincide with the origin O of the real space coordinate system) and an x coordinate 
xc(N - 1 ) obtained a time period AT earlier. If the following expression (1 2) is satisfied, it is determined that there is a 
high probability of collision. 



50 



55 



f^yW-i;^] ^xc(0)-xc(N-l) 

= 2 IZv(O) Zv(N-l)j 



(12) 
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[0092] As shown in FIG. 1 9, If an animal 20 comes in a direction approximately orthogonal to the direction of running 
of the automotive vehicle 10. when Xv(N - 1)/Zv(N - 1) = Xv(0)/Zr(0) holds, in other words, when a ratio VpA/s = Xr(N 
- 1)/Zr(N - 1) holds (Vp is a speed of the animal 20 and Vs is a relative speed of the vehicle 10), an azimuthal angle G 
d of the animal 20 as viewed from the vehicle 10 is constant, in this case, there is a high probability of collision. The 

5 expression (12) is used for determining the probability by taking the width a of the automotive vehicle 10 into account. 
In the following, a method of deriving the expression (12) will be described with reference to FIG. 20. 
[0093] When a straight line extending through the latest position coordinates of the object 20 and the position coor- 
dinates thereof obtained a time period AT earlier, that is, the approximate straight line LMV, intersects an X-Y plane 
(plane including the X axis and the Y axis, i.e. a plane which contains a line (X axis) corresponding to the front end of 

10 the vehicle 1 0 and extends in a direction perpendicular to the direction of running of the vehicle 1 0) Jf the X coordinate 
of the intersection point thereof Is represented by XCL, collision occurrence conditions in which the width a of the 
vehicle 10 is taken into account are given by the following expression (13): 



15 



-a/2 ^ XCL ^ aJ2 (13) 

[0094] On the other hand, a straight line formed by projecting the approximate straight line LMV on an X-Z plane Is 
given by the following equation (14): 

[0095] When Z = 0, X = XCL are substituted into this equation to calculate XCL, the following equation (1 5) is obtained : 

^5 _ Xv(0)/Zv(0)-Xv(N-1)/Zv(N-1) 

1/Zv(0)-1/Zv(N-1) ^ 

[0096] Further, since the real space coordinate X and the coordinate xc on the image have a relationship shown by 
the above equation (3), the following equations (16) and (17) hold: 

30 

Xv(0) = xc(0) X Zv(0)/f (1 6) 

Xv(N - 1 ) = xc(N - 1 ) X Zv(N-1 )/f (1 7) 

[0097] When the above equations (16) and (1 7) are applied to the equation (1 5), the X coordinate XCL of the inter- 
section point Is given by the following equation (18): 

40 XC(0)/f-XC(N-1)/f .^gv 

(18)1/Zv(0)-1/Zv(N-1) ^ ' 

By substituting the XCL into the expression (13), the conditions defined by the above expression (12) are obtained. 
[0098] Referring again to FIG. 4, when it is determined at step 43 that there is a high probability of collision, the 
45 program proceeds to step S44, whereas when it is detenmined that there is not a high probability of collision, the warning 
detennination process is terminated. 

[0099] At step S44, a warning output detennination process is carried out, for determining whether or not a warning 
should be issued. First, it is determined from an output signal from the brake sensor whether or not the driver is carrying 
out braking operation; i.e, operating the brake of the vehicle 1 0. If he is not operating the brake, the program immediately 

50 proceeds to step S45 to issue a warning. If he is operating the brake, deceleration Gs generated by the braking operation 
is calculated (as a positive value). When the deceleration Gs is equal to or smaller than a predetemriined threshold 
GTH, the program proceeds to step S46, whereas when Gs > GTH holds, it is detennined that collision can be avoided 
by the braking operation, followed by temninating the warning detennination process. This makes it possible to sub- 
stantially prevent the driver from being warned when he is carrying out an appropriate braking operation, thereby 

55 preventing the driver from being annoyed by unnecessary warnings. 

[0100] The predetemiined threshold GTH is set according to the following equation (19): 



35 
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GTH = (19) 
2XZV(0) 

5 [0101] The threshold thus determined corresponds to a condition for stopping the vehicle 1 0 at a distance of travel 
equal to or smaller than the distance Zv(0), assuming that the deceleration Gs generated by the braking operation is 
maintained. 

[0102] At step S45, a voice alann is generated by the speaker 3, and as shown in FIG. 21 B, an image obtained e. 
g. by the camera 1 R is displayed on the screen 4a of the HUD 4 such that a closing object is emphatically displayed 
10 (for instance, enclosed in a frame for emphasis). FIG. 21 A shows a state where the screen 4a is not displayed, while 
FIG. 21 B shows a state where the screen 4a is displayed. This enables the driver to positively recognize an object 
having a high probability of collision against the vehicle 10. 

[0103] As described above, in the present embodiment, the probability of collision is determined on condition that 
the expressions (10a) and (10b) are satisfied, and the warning is issued according to the position and the relative 
15 movement vector of the object Therefore, even if a detection error of the relative speed is comparatively large, the 
probability of collision is accurately detennlned, and the warning can be Issued at a desired timing. Further, other 
vehicles that are running in the opposite lane or running ahead In the same lane are excluded from the objects of 
warning by the condition of the expression (10a). Accordingly, frequent unnecessary warnings can be substantially 
prevented. 

20 [0104] Further, positions of an identical object in the real space are calculated based on a plurality of time series 
items of position data of the object, and the movement vector of the object is obtained based on its positions In the 
real space. Then, the probability of collision between the object and the automotive vehicle 10 is determined based on 
the movement vector thus calculated. Therefore, using the system hereof, differently from the conventional system, it 
is possible to substantially prevent the occun-ence of an erroneous determination, and thereby enhance accuracy of 

25 detemnination of the probability of collision. 

[0105] Further, the approximate straight line LMV approximating the locus of relative movement of an object to the 
automotive vehicle 10 is calculated, and the position coordinates of the object are connected such that positions of the 
detected object are on the approximate straight line, and the movement vector of the object is obtained based on the 
corrected position coordinates. This makes it possible to reduce adverse influence of errors in detecting the positions 

30 of the object, and to more accurately detemnine the probability of collision. 

[0106] Further, since determination of a potential collision is earned out by taking the width a of the vehicle 1 0 into 
account, it is possible to more accurately detemnine the probability of collision and prevent the driver from being un- 
necessarily warned. 

[0107] In the present embodiment, the image-processing unit 2 constitutes distance calculating means, relative 
35 speed calculating means, relative position -detecting means, movement vector-calculating means, determining means, 
and part of warning means. More specifically, steps S31 to S33 in FIG. 3 correspond to the distance calculating means. 
The calculation of the equation (9) corresponds to the relative speed calculating means. Steps SI 4 to 819 In FIG. 3 
correspond to the relative position-detecting means. Steps 820 to S23 in the figure correspond to the movement vector- 
calculating means. Step S41 to S44 in FIG. 4 correspond to the detemnining means. Step S45 in FIG. 4 and the speaker 
40 3 and the HUD 4 correspond to the warning means. 

[0108] Further, the image-processing unit 2 constitutes a distance calculating module, a relative speed calculating 
module, a relative position-detecting module, a movement vector-calculating module, a probability determining module, 
and part of a warning module. More specifically, steps S31 to S33 in FIG. 3 correspond to the distance calculating 
module. The calculation of the equation (9) corresponds to the relative speed calculating module. Steps SI 4 to S19 
45 in FIG. 3 correspond to the relative position-detecting module. Steps S20 to S23 in the figure correspond to the move- 
ment vector-calculating module. Step S41 to S44 in FIG. 4 correspond to the probability determining module. Step S45 
in FIG. 4 and the speaker 3 and the HUD 4 correspond to the warning module. 

[0109] It should be noted that the invention is not limited to the above embodiment, but various variations and mod- 
ifications are possible. For instance, although in the present embodiment, infrared cameras are used as imaging means, 
50 this is not limitative, but TV cameras capable of detecting only nomnal visible rays, as disclosed in Japanese Laid-Open 
Patent Publication (Kokai) No. 9-226490, may be employed. By using infrared cameras, however, the extraction process 
for extracting an animal or a running vehicle can be simplified, and the system can be realized by using an arithmetic 
unit having relatively low performance. 

[01 10] Further, although in the above embodiment, an example which monitors a front portion of the environment of 
55 an automotive vehicle is shown, this is not limitative, but the system may be configured such that any portions of the 

environment in any directions, including a rear portion thereof, can be monitored. 

[01 1 1 ] It is further understood by those skilled in the art that the foregoing is a preferred embodiment of the invention, 
and that various changes and modifications may be made without departing from the spirit and scope thereof. The 
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scope of the invention is indicated by the appended claims. 

[0112] A vehicle environment monitoring system for an automotive vehicle, is capable of accurately detecting the 
movement of an object existing in an external environment of the vehicle, and detemiining the probability of collision 
between the object and the vehicle, thereby appropriately waming the driver. A running speed of the vehicle is detected, 
and a distance between the object and the vehicle is calculated, based on the Image obtained by the imaging device. 
A relative speed between the object and the vehicle is calculated according to the calculated distance. It is detenmined 
whether or not there Is a significant probability that the vehicle may collide with the object, based on the running speed 
of the vehicle and the distance therebetween on condition that the relative speed is in the vicinity of the running speed 
of the vehicle. 

Claims 

1 . A vehicle environment monitoring system for monitoring an extemal environment proximate to a vehicle, and for 
15 detect an object existing in the external environment of said vehicle from an image obtained by imaging means 

mounted on said vehicle, the vehicle environment monitoring system comprising: 

vehicle speed detecting means for detecting a running speed of said vehicle; 

distance calculating means for calculating a distance between said object and said vehicle based on the image 
20 obtained by said imaging means; 

relative speed calculating means for calculating a relative speed between said object and said vehicle accord- 
ing to the distance calculated by said distance calculating means; and 

probability detennlning means for detemnlnlng whether or not there is a significant probability that said vehicle 
may collide with said object, based on the running speed of said vehicle and the distance calculated by said 
25 distance calculating means, on condition that the relative speed is in the vicinity of the running speed of said 

vehicle. 

2. A vehicle environment monitoring system according to claim 1 , wherein said probability determining means carries 
out the determination, when an absolute value of a difference between the relative speed and the running speed 

30 of said vehicle is less than or equal to half of the running speed of said vehicle. 

3. A vehicle environment monitoring system according to claim 1 , further comprising 

relative position-detecting means for detecting a relative position of said object to said vehicle, based on the 
image obtained by said imaging means and the distance calculated by said distance calculating means, to thereby 
35 obtain position data; and 

movement vector-calculating means for calculating positions of said object in a real space based on a plurality 
of time series Items of the position data detected on said object by said relative position-detecting means, and 
calculating a movement vector of said object based on the positions in said real space; 

wherein said probability detemriining means detemnines whether or not the probability of collision is high 
40 based on the movement vector, when it Is detenmined based on the running speed of said vehicle and the distance 

that there is a probability of collision. 

4. A vehicle environment monitoring system according to claim 3, wherein said movement vector-calculating means 
includes 

45 approximate straight line-calculating means for calculating an approximate straight line approximating a locus 

of relative movement of said object, and 

position data-correcting means for connecting the time series Items of the position data by using the approx- 
imate straight line, 

said movement vector-calculating means calculating the movement vector based on the corrected time series 
50 items of the position data. 

5. A vehicle environment monitoring system according to claim 1 , wherein said probability determining means carries 
out the determination by applying collision detemiination conditions that are dependent on a width of said vehicle. 

55 6. A vehicle environment monitoring system according to daim 4, wherein said approximate straight line-calculating 
means calculates the approximate straight line as a three-dimensional straight line including data of a height of 
said object. 
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7. A vehicle environment monitoring system according to claim 1 , wherein said imaging means comprises two infrared 
cameras capable of detecting infrared rays. 

8. A vehicle environment monitoring system according to claim 1 , wherein said imaging means comprises two TV 
5 cameras for detecting infrared rays or visible rays, and wherein said relative position -detecting means includes 

search area-setting means for setting, based on a position of an object image contained in an image output 
from one of said two TV cameras, a search area within an image output from the other of said two TV cameras 
for searching for a corresponding object image contained in the image output from the other of said two TV cameras; 
and 

10 corresponding object image-identifying means for identifying the corresponding object image by carrying out 

a correlation operation on data within said search area, 

said distance-calculating means calculating the distance between said vehicle and said object based on a 
parallax between the object image and the corresponding object image. 

IS 9. A vehicle environment monitoring system according to claim 1 , further comprising warning means for warning a 
driver of said vehicle, when it is detemnined by said probability determining means that the probability of collision 
against said object is high, 

wherein said warning means is inhibited from warning the driver when the driver is carrying out a braking 
operation and at the same lime deceleration caused by the braking operation is larger than a predetermined thresh- 
20 old. 

10. A vehicle environment monitoring method for monitoring an external environment proximate to a vehicle, in which 
an object existing in the external environment of said vehicle is detected from an image obtained by an imaging 
device mounted on said vehicle, said method comprising the steps of: 

25 

a) detecting a running speed of said vehicle; 

b) calculating a distance between said object and said vehicle based on the image obtained by said imaging 

device; 

c) calculating a relative speed between said object and said vehicle according to the calculated distance; and 
30 d) determining whether or not there is a significant probability that said vehcle may collide with said object, 

based on the running speed of said vehicle and the calculated distance, on condition that the relative speed 
is in the vicinity of the running speed of said vehicle. 

11. A vehicle environment monitoring method according to claim 10, wherein the probability detemriination is carried 
35 out, when an absolute value of a difference between the relative speed and the running speed of said vehicle is 

less than or equal to half of the running speed of said vehicle. 

12. A vehicle environment monitoring method according to claim 10, further comprising the steps of 

40 e) detecting a relative position of said object to said vehicle based on the image obtained by said imaging 

device and the calculated distance, to thereby obtain position data; 

f) calculating positions of said object in a real space based on a plurality of time series items of the position 
data detected on said object; 

g) calculating a movement vector of said object based on the positions in said real space; and 

45 h) detemnlning whether or not the probability of collision is high based on the movement vector, when it is 

determined based on the running speed of said vehicle and the distance that there is a probability of collision. 

13. A vehicle environment monitoring method according to claim 12, wherein said step of cateulating the movement 
vector includes the steps of: 

so 

j) calculating an approximate straight line approximating a locus of relative movement of said object; and 
k) correcting the time series items of the position data by using the approximate straight line, 

said movement vector being calculated based on the corrected time series items of the position data. 

55 14. A vehicle environment monitoring method according to claim 10, wherein the probability detemriination is carried 
out by applying collision determination conditions that are dependent on a width of said vehicle. 

1 5. A vehicle environment monitoring method according to claim 1 3, wherein the approximate straight line is calculated 
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as a three-dimensional straight tine including data of a height of said object. 

16. A vehicle environment monitoring nnethod according to claim 10, wherein said imaging device comprises two in- 
frared cameras capable of detecting infrared rays. 

5 

17. A vehicle environment monitoring method according to claim 10, wherein said imaging device comprises two TV 
cameras for detecting infrared rays or visible rays, and said step of detecting a relative position of said object to 
said vehicle includes the steps of: 

10 I) setting, based on a position of an object image contained in an image output from one of said two TV cameras, 

a search area within an image output from the other of said two TV cameras for searching for a corresponding 
object image contained in the image output from the other of said two TV cameras; and 
m) identifying the corresponding object image by carrying out a congelation operation on data within said search 
area, 

15 

the distance between said vehicle and said object being calculated based on a parallax between the object 
image and the corresponding object image. 

18. A vehicle environment monitoring method according to claim 10, further comprising the step of warning a driver 
20 of said vehicle, when it is determined that the probability of collision against said object is high, 

wherein the step of warning is omitted when the driver is carrying out a braking operation and at the same 
time deceleration caused by the braking operation is larger than a predetermined threshold. 

19. A vehicle environment monitoring system for monitoring an extemal environment proximate to a vehicle, and for 
25 detecting an object existing in an environment of said vehiclef rom an image obtained by an imaging device mounted 

on said vehicle, the vehicle environment monitoring system comprising: 

a vehicle speed sensor for detecting a running speed of said vehicle; 

a distance calculating module for calculating a distance between said object and said vehicle based on the 
30 image obtained by said imaging device; 

a relative speed calculating module for calculating a relative speed between said object and said vehicle ac- 
cording to the distance calculated by said distance calculating module; and 

a probability determining module for determining whether or not there is a significant probability that said 
vehicle may collide with said object, based on the running speed of said vehicle and the distance calculated 
35 by said distance calculating module, on condition that the relative speed is in the vicinity of the running speed 

of said vehicle. 

20. A vehicle environment monitoring system according to claim 1 9, wherein said probability determining module car- 
ries out the detemni nation, when an absolute value of a difference between the relative speed and the running 

40 speed of said vehicle is less than or equal to half of the running speed of said vehicle. 

21. A vehicle environment monitoring system according to claim 19, further comprising 

a relative position -detecting module for detecting a relative position of said object to said vehicle based on 
the image obtained by said imaging device and the distance calculated by said distance calculating module, to 
45 thereby obtain position data; and 

a movement vector-calculating module for calculating positions of said object in a real space based on a 
plurality of time series items of the position data detected on said object by said relative position -detecting module, 
and calculating a movement vector of said object based on the positions in said real space; 

wherein said probability determining module detenmines whether or not the probability of collision is high 
so based on the movement vector, when it is determined based on the running speed of said vehicle and the distance 

that there is a probability of collision. 

22. A vehicle environment monitoring system according to claim 21 , wherein said movement vector-calculating module 
includes 

55 approximate straight line-calculating module for calculating an approximate straight line approximating a 

locus of relative movement of said object, and 

position data-correcting module for correcting the time series items of the position data by using the approx- 
imate straight line. 
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said movement vector-calculating module calculating the movement vector based on the corrected time 
series items of the position data. 



23. A vehicle environment monitoring system according to claim 1 9. wherein saUJ f^'^^^'^V ^^n^^^^^ 
5 ries out the determination by applying collision detennination conditions that are dependent on a width of said 

vehicle. 

OA A vehicle environmentmonitoring system according to claim 22, wherein said approximate straight line-calculating 
mrdu^e caTcu~e^^^^^^^ line as a three-dimensional straight line including data of a height of 

10 said object. 

25. A vehicle environment monitoring system according to claim 19, wherein said Imaging device comprises two in- 
frared cameras capable of detecting infrared rays. 

26 A vehicle environment monitoring system according to claim 19, wherein said imaging device comprises two W 
Cameras fordetecting infrared rays or visible rays, and wherein said relative position-detecting module includes 

ea;chXsettingmodureforsettlng,bas 
from one of said two TV cameras, a search area within an image output from the other of said two TV cameras 
iLrTe^chingforacorrespondingobjectim^^^ 

^""^ a corresponding object image-identifying module for identifying the corresponding object image by carrying 
out a correlation operation on data within said search area, ^ i.- , o 

saJd distance calculating module calculating the distance between said vehicle and said object based on a 
parallax between the object image and the corresponding object »mage. 
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20 and 



25 



30 



27 A vehicle environment monitoring system according to claim 19, further comprising a warning "^^'^"'^'^^^["'"a 
a driver of said vehicle, when it is detemnined by said probability detemiining module that the probability of collision 

''"thel'in^sa'S w'aSg module is inhibited from warning the driver when the driver is carrying out a baking 
operaJ^^nLTdat^e sameLe deceleration caused by t 
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